Stochastic and Adversarial Online Learning without Hyperparameters

Most online optimization algorithms focus on one of two things: performing well in adversarial settings by adapting to unknown data parameters (such as Lipschitz constants), typically achieving O( √ T ) regret, or performing well in stochastic settings where they can leverage some structure in the losses (such as strong convexity), typically achieving O(log(T )) regret. Algorithms that focus on the former problem hitherto achieved O( √ T ) in the stochastic setting rather than O(log(T )). Here we introduce an online optimization algorithm that achieves O(log(T )) regret in a wide class of stochastic settings while gracefully degrading to the optimal O( √ T ) regret in adversarial settings (up to logarithmic factors). Our algorithm does not require any prior knowledge about the data or tuning of parameters to achieve superior performance. 1 Extending Adversarial Algorithms to Stochastic Settings The online convex optimization (OCO) paradigm [1, 2] can be used to model a large number of scenarios of interest, such as streaming problems, adversarial environments, or stochastic optimization. In brief, an OCO algorithm plays T rounds of a game in which on each round the algorithm outputs a vector wt in some convex space W , and then receives a loss function `t :W → R that is convex. The algorithm’s objective is to minimize regret, which is the total loss of all rounds relative to w, the minimizer of ∑T t=1 `t in W : RT (w ) = T ∑